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METHOD AND SYSTEM FOR INTERACTIVE 
MOTION TRAINING AND A USER INTERFACE 
THEREFOR 

CROSS REFERENCE TO RELATED APPLICATIONS 

This application claims the benefit of priority from the prior provisional 
application entitled "METHOD AND SYSTEM FOR INTERACTIVE MOTION 
TRAINING AND A USER INTERFACE THEREFOR", serial number 60/146,933, filed in 
the United States Patent and Trademark Office on July 30, 1999. 

FIELD OF THE INVENTION 

The present invention relates generally to a process for capturing three- 
dimensional motions of an instructor, actor or other demonstrator and converting the three- 
dimensional motion information into a three- or two-dimensional digital display format; 
developing a digital character including equipment such as clothing and gear appropriate 
for the motion being taught and having the same physical characteristics, height, weight, 
and shape of the instructor; dividing the full training motion sequence into individual 
motion segments; compressing the motion segments into a single, digitized software 
animation unit; and adding textual, audio or visual explanatory material associated with a 
particular motion segment to the animation unit. The present invention also relates 
generally to a computer software program that retrieves the animation unit from a remote 
file server or a local database or other location; and controls the animation unit to display 
the training motion, individual motion segments and/or the explanatory material on a 
display, or, for example, an Internet browser, television, or any hand-held or other 
computer device capable of running the software program and providing a display. 

BACKGROUND OF THE INVENTION 

Motion training is used to teach millions of people how to perform or perfect 
a new technique. The method of learning a motion most often employed by those seeking 
to learn new techniques is through self-education using books or videotapes or other 
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educational material. For example, students may purchase instructional books or video 
tapes to read or review the proper techniques performed by an instructor, and then proceed 
to perform the motions on their own using the guidelines provided by the material. These 
methods, however, are static and typically do not allow for the student to interact or 
5 control the way in which motions are displayed. Also, the student may not gain the benefit 
of seeing the same motion from different vantage points or camera views. Furthermore, 
for the average person, this process can be difficult and unrewarding because there is 
typically no review of whether the student's technique was performed accurately, and 
therefore often leads to students improperly learning the motions. 

10 Motion training can also be taught by an instructor verbally directing the 

student to recognize the desired positions and sequence of the motions by imitation, feel 
and through the explanatory comments of the instructor. To receive the most money for 
their time, the instructor usually assembles a group of people, or class, and performs the 
techniques for many people at once. This method of learning can be too costly and time 

15 restrictive for the student as he or she is subject to the time schedule and location at which 
the instructor teaches the techniques. And, if the student wants to repeat any of the 
techniques with the instructor, or have the benefit of one-to-one instruction, an additional 
fee is charged to the student above and beyond the cost of the original lesson plans 1 . 
Furthermore, the motions of the instructor are presented in one speed - as quickly as the 

20 instructor naturally moves; in one temporal direction - forward in time; and usually in one 
complete sequence. The segments or subsequences which may comprise the sequence of 
an activity are not easily viewed or separated by the instructor nor by a student viewing a 
live demonstration. 

What is desired is a method and software system for training motions to a 
25 student which provides a digitally displayed character wearing appropriate equipment such 
as clothing and gear for the motion to be taught, where the character resembles an actual 
instructor who has performed the motion, possibly in accordance with sanctioned script 
instructions before a motion capture set up; and where the student may select individual 
motion segments comprising the complete motion sequence to display and redisplay along 
30 with explanatory material associated with that segment or subsequence; and where the 
student may select one of several camera angles for viewing the motion; and where the 

2 
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student may select individual portions of the digital character display to view additional 
information related to that portion. 

SUMMARY OF THE INVENTION 

Generally speaking, in accordance with the present invention, a method and 
5 system are presented for condensing one or more three dimensional, digital animation 
motion sequences into a self-contained, digitized animation unit, and dividing the motion 
sequences into separate motion segments or subsequences associated with explanatory text, 
audio, graphic or other material; and for providing an interactive user-interface for 
displaying and controlling the animation unit. The separate motion segments or 
10 subsequences of the overall animation sequence may be played as individual self-contained 
units by a user, allowing a user to focus on one particular portion of a physical activity 
(e.g., only the ball toss subsequence in a tennis serve sequence). The user may play such a 
subsequence so that the motion display starts at the beginning of the subsequence, and not 
before, and ends at the end of the subsequence, and not after. Further, explanatory 
15 material associated with that segment may only be presented while the user is viewing that 
motion segment. Thus other portions or subsequences of the animation do not interfere 
with the user's study of the subsequence. 

The present invention provides for a method of capturing the dynamic 
motion techniques of an instructor performing a motion, preferably using a script 

20 sanctioned or approved by an appropriate organization, digitizing the motion into data files, 
superimposing a character graphic on the three-dimensional animation data, converting the 
data into three- or two-dimensional vector data, cleaning and filtering the animation data to 
smooth the appearance and highlight those portions of the character which are most 
relevant to instruction of the technique, dividing the motion sequence into individual 

25 motion segments, adding explanatory material associated with each individual segment, and 
efficiently delivering the animation unit to a student using an interactive control interface 
over a communication network, the Internet, television, or any hand-held computer device 
capable of running an interactive user interface for controlling the animation unit. 
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The present invention also provides for creating a simple, inexpensive and 
easy to use motion training user-interface, providing instructional information to a user, 
which displays a vector graphical representation or animation of a motion technique of an 
instructor who has correctly performed the motion sequence, and which also provides 
5 audio, text, or other material and Internet hyperlinks associated with one or more digitized 
motion segments or portions of the sequence, or associated with a portion of the digital 
character representation in an interactive environment. The software display program and 
user interface uses the animation unit as input. The animation unit may also include an 
entire course of training motion sequences performed by an instructor dynamically 
10 performing particular motion techniques. 

The present invention further provides for a software display program or 
player, capable of displaying the motion techniques from an animation unit on an Internet 
browser, television, or any hand-held computer device capable of running the software 
player. The software player accepts and processes user input to control the digitized 

15 animation motion sequence, and individual motion segments, the speed of animation 
playback, and one or more different camera views of the same motion sequence. The 
software player typically also allows the user to display and review the animation motion 
sequences and individual segments to help the student isolate the different motions that 
make up the complete technique and provide additional explanatory material associated 

20 with certain portions of the display. 

Accordingly, it is an important object of this invention to provide a method 
and system for interactive motion training including converting three dimensional training 
motion sequences performed by an instructor to three- or two- dimensional vector graphic 
data; including explanatory text, audio, graphical or other material associated with an 
25 individual motion segment of the training motion sequence; and providing an interactive 
user interface for displaying and controlling the training motion sequence and individual 
motion segments while viewing the explanatory material. 

Another object of the invention is to provide a method and system for 
training a user to perform a motion sequence including a three- or two dimensional vector 
30 graphic representation of an instructor performing the motion sequence, and an interactive 
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user display allowing the user to select specific motion segments from the training motion 
sequence to review, which motion segments are associated with explanatory material also 
displayed on the user interface. 

A further object of the invention is to provide a user interface for displaying 
5 three- or two- dimensional vector graphic representations of a training sequence and 
explanatory material associated with individual segments of the training sequence motion 
wherein the user may select one of several camera angles with which to view the motion 
segments, and may select individual portions of the representation whereby the user is 
shown additional information associated with that portion of the representation. 

10 Still another object of the invention is to provide a user interface for 

displaying training motion sequences in which the user may select individual motion 
segments for display, may freeze, reverse or fast forward individual frames or a range of 
frames from the training motion sequence, and may view additional material associated 
with an individual frame or range of frames or associated with a user selected portion of 

15 the representation. 

Still a further object of invention is to provide a digitized, sanctioned and 
approved motion training sequence as performed by an experienced instructor and 
explanatory materials associated with individual motion segments of the sequence; and a 
software program for controlling and displaying the sequence, whereby the student may 
20 control the camera angle for viewing the sequence, and a frame or range of frames 
comprising a portion of the training sequence or an individual motion segment. 

Yet another object of the invention is to provide a user interface displaying 
an animation unit including individual motion segments forming a motion sequence; and 
displaying explanatory text associated with a frame or range of frames of the motion 
25 sequence; and further providing additional information, such as audio, graphic or other 
material, in response to user interaction. 

Yet a further object of the invention is to provide a system and method for 
easily teaching a motion technique by allowing a student to view a motion sequence broken 
down into individual motion segments, wherein the motion sequence is generated from an 
30 instructor performing a script approved or sanctioned by an appropriate organization and 
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creating a vector graphic representation, and providing explanatory material associated 
with portions of the motion sequence. 

Still other objects and advantages of the invention will in part be obvious and 
will in part be apparent from the following detailed specification. 

5 The invention accordingly comprises the several steps and the relation of one 

or more of such steps with respect to each of the others, the system embodying features of 
construction, combinations of elements and arrangement of parts which are adapted to 
effect such steps, and the interface embodying features of construction, combinations of 
elements and arrangement of parts which are adapted to allow user interaction therewith, 
10 all as exemplified in the following detailed disclosure. 

BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present invention and its 
associated advantages, reference is now made to the following descriptions taken in 
conjunction with the accompanying drawings, in which: 

15 Fig. 1 is a block diagram of the major elements of one implementation of the 

present invention; 

Fig. 2 A is a block diagram of the individual elements that make up the 
major element 102 in Fig. 1; 

Fig. 2B is a block diagram of the individual elements that make up the major 
20 element 106 in Fig. 1; 

Fig. 2C is a block diagram of the individual elements that make up the major 
element 110 in Fig. 1; 

Fig. 2D is a block diagram of the individual elements that make up the 
major element 114 in Fig. 1; 

25 Fig. 3 is a diagram illustrating the individual instructional segments that 

make up an animation sequence to be displayed on a monitor according to one embodiment 
of the present* invention; 
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Fig. 4 A is the main control user-interface illustrating a number of animation 
sequences, with level of expertise, as it can be displayed on a monitor according to one 
embodiment of the present invention; 

Fig. 4B is the main control user-interface illustrating the product marketing 
5 introduction sequence as it can be displayed on a monitor according to one embodiment of 
the present invention; 

Fig. 5 A is the main control user-interface illustrating one frame of the 
starting segment of the animation motion sequence from a side view as it can be displayed 
on a monitor according to one embodiment of the present invention; 

10 Fig. 5B is the main control user-interface illustrating another frame from the 

start segment of the animation motion sequence from a side view as it can be displayed on 
a monitor according to one embodiment of the present invention; 

Fig. 5C is the main control user- interface illustrating a third frame from the 
start segment of the animation motion sequence from a side view as it can be displayed on 
15 a monitor according to one embodiment of the present invention; 

Fig. 6 is the main control user-interface illustrating the last frame from the 
first individual instructional segment and highlighted areas of focus as it can be displayed 
on a monitor from a side camera view according to one embodiment of the present 
invention; 

20 Fig. 7 A is the main control user-interface illustrating the last frame from the 

second individual instructional segment and highlighted area of focus as it can be displayed 
on a monitor from a side camera view according to one embodiment of the present 
invention; 

Fig. 7B is the main control user-interface illustrating the last frame from the 
25 second individual instructional segment and highlighted area of focus as it can be displayed 
on a monitor from a front camera view according to one embodiment of the present 
invention; 
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Fig. 8 A is the main control user-interface illustrating the first frame from the 
third individual instructional segment as it can be displayed on a monitor from a side 
camera view according to one embodiment of the present invention; 

Fig. 8B is the main control user-interface illustrating a latter frame from the 
5 third individual instructional segment as it can be displayed on a monitor from a side 
camera view according to one embodiment of the present invention; 

Fig. 8C is the main control user-interface illustrating still a later frame from 
the third individual instructional segment as it can be displayed on a monitor from a side 
camera view according to one embodiment of the present invention; 

10 Fig. 8D is the main control user-interface illustrating the last frame from the 

third individual instructional segment and highlighted areas of focus as it can be displayed 
on a monitor from a side camera view according to one embodiment of the present 
invention; 

Fig. 8E is the main control user-interface illustrating the last frame from the 
15 third individual instructional segment and highlighted areas of focus as it can be displayed 
on a monitor from a front camera view according to one embodiment of the present 
invention; 

Fig. 9 A is the main control user-interface illustrating the last frame of the 
final individual instructional segment and highlighted areas of focus as it can be displayed 
20 on a monitor from a side camera view according to one embodiment of the present 
invention; 

Fig. 9B is the main control user-interface illustrating the individual 
highlighted area of focus on the final individual instructional segment as it can be displayed 
on a monitor from a side camera view according to one embodiment of the present 
25 invention; 

Fig. 9C is the main control user-interface illustrating another individual 
highlighted area of focus on the final individual instructional segment as it can be displayed 
on a monitor from a side camera view according to one embodiment of the present 
invention; 
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Fig. 10A is the main control user-interface illustrating the ending 
instructional segment as it can be displayed on a monitor according to one embodiment of 
the present invention; 

Fig. 10B illustrates an additional instructional topic as it can be displayed on 
5 a monitor according to one embodiment of the present invention; 

Fig. 10C illustrates another additional instructional topic as it can be 
displayed on a monitor according to one embodiment of the present invention; 

Fig. 11A is the main control user-interface describing the equipment used 
for the instructional sport with highlighted areas of focus as it can be displayed on a 
10 monitor from a side camera view according to one embodiment of the present invention; 

Fig. 11B illustrates the textual description for the individual highlighted area 
of focus for an equipment item used for the instructional sport as it can be displayed on a 
monitor according to one embodiment of the present invention; 

Fig. 11C illustrates the textual description for an individual highlighted area 
15 of focus for another equipment item used for the instructional sport as it can be displayed 
on a monitor according to one embodiment of the present invention; 

Fig. 12 is a block diagram illustrating the hierarchy and navigational flow 
for the instructional content according to one embodiment of the present invention; 

Fig. 13 is a block diagram illustrating the individual components of the 
20 animation unit according to one embodiment of the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

In the following description, various aspects of the present invention will be 
described. For purposes of explanation, specific configurations and details are set forth in 
order to provide a thorough understanding of the present invention. However, it will also 
25 be apparent to one skilled in the art that the present invention may be practiced without the 
specific details. Furthermore, well known features may be omitted or in the description 
simplified in order not to obscure the present invention. 
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The present invention is utilized for providing instructional information 
including motion training techniques to a user; in particular, motion training for various 
sports, physical therapy, dance, workplace or recreational activities. The present invention 
is particularly useful in training a motion sequence in which the positions of the body and 
5 its limbs, as well as sport, therapy, work, or recreational equipment, are to be compared 
during the motion sequence and emulated by the student. 

In a sporting activity, a participant who is trained in the proper motion 
technique may be able to improve his or her skill level or avoid an injury from performing 
a repetitive motion accurately. Physical therapy can be more effective when exercising a 
10 specific muscle needing therapy through a particular motion when that motion technique is 
performed correctly. In the workplace, a worker's safety may be protected or a gain in 
production efficiency may result from proper technique in performing a particular motion. 

In many repetitive motion activities the larger muscles of the body typically 
do most of the work. Golf, tennis, baseball, football, basketball, running, and many 

15 physical therapies, to name a few, are based on fundamental movements and techniques 
that typically result from the correct positioning of the body as a whole. Correct motion 
technique and placement of the major body parts with respect to one another is of major 
importance to the successful completion of the intended action. Even where the perceived 
motion action (i.e. , throwing a baseball) is performed more by the joints or extremities 

20 than the larger body parts, it still is usually the proper sequence of movements of the large 
muscles of the body which bring the joints and extremities into correct alignment with the 
appropriate timing of execution. 

In fact, for proper results in successfully performing the technique in many 
motions of sports such as golf, tennis, and baseball, where the object is to hit a ball and the 

25 like, it is actually more important for the extremities to be in relatively correct alignment, 
through proper positioning of the larger muscles of the body moving in ways that are subtly 
counterproductive to maximum impact. Incorrect motion existing solely in the movements 
of the joints and extremities (i.e. , poorly timed rolling of the forearms in a golf swing) is 
more clearly demonstrated to the student with the present invention, which can interactively 

30 focus on a particular portion of the movement unlike with books or videotapes. The 



10 



WO 01/09861 



PCT/US00/20752 



student can see the actual shapes of the limbs in question moving in concert with one 
another and can highlight, through interaction with the display as described further below, 
the position of the extremity. That is, because the motion display sequence has certain 
highlighted areas, the student viewing a segment or subsequence may focus on a detailed 
5 view of the portion of the display highlighted to learn the correct positioning. 

Although the present invention will now be described with regard to a 
particular in-line skating stop, this is but one example. The present invention may also be 
used for, among other techniques, training strokes or serves in tennis, hitting or pitching in 
baseball, running, hitting a nail with a hammer, proper technique to pickup heavy objects, 
10 and the like. Furthermore, the system and method of the present invention may be used for 
training in any sort of physical activity, such as physical therapy or dance. 

The following embodiments are described in sufficient detail to enable those 
skilled in the art to practice the invention, and it is to be understood that other 
embodiments may be utilized and that structural, logical, and graphical changes may be 
15 made without departing from the spirit and scope of the present invention. 

The approval of a governing body of representatives in the field in which the 
technique resides, or other appropriate organization, may be consulted and a sanctioned 
technique will be developed. A script may be written for an instructor who performs the 
technique in the approved way. When used herein, the term "instructor" may mean any 
20 person demonstrating a physical activity, and is not limited to a person who is a certified 
trainer or a professional in a given field. 

Fig. 1 is a block diagram of the major elements of one implementation of the 
present invention. Referring to Fig. 1, in step 102, motion capture cameras surrounding 
the area of movement detect light reflecting from optical balls positioned on the instructor 

25 and motion capture software digitizes the three-dimensional movements into data points. In 
step 104, the software further generates HTR motion data from the data points captured. It 
is well known to capture the motion of an actor wearing devices such as optical balls or 
lights indicating the relative motion of the actor's body parts; other methods of motion 
capture may be used with the system and method of the present invention. The creation 

30 and use of HTR file data is well known. 
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In step 106, a three-dimensional character is created which typically includes 
all equipment, including equipment such as safety gear and clothing used by people who 
perform the motion technique, in a computer software program. The HTR motion data is 
applied to the three-dimensional character model, which creates a three-dimensional 
5 version of the motion sequence. To optimize the viewing of the motion sequences on an 
on-line interactive display such as a browser, an additional computer software program 
may be installed on the computer to allow the main software program to call individual 
functions of the install program. This type of linked program is known as a plug-in. 

In step 108, a plug-in program is used to render the three-dimensional 
10 motion sequences into two-dimensional vector data. One example of such a computer 
software program is 3D Studio Max running on a Microsoft Windows operating system; 
other software programs or modules may be used. A preferred minimum configuration 
supporting such a program includes a 128 Mbytes of main memory and an Intel Pentium 
333 MHz processor. Those skilled in the art will recognize, however, that other 
15 comparable hardware and software programs may be employed without diverting from the 
spirit or scope of the present invention. Furthermore, where bandwidth is not an issue, the 
invention may be performed using a three-dimensional representation. 

In step 110, the two-dimensional vector data files are imported into a vector- 
based animation software program. One such software program is Macromedia's Flash 

20 development environment; other software programs may be used. In step 112, animation 
data is created, and in step 1 14 an interactive animation unit or an animation data file is 
created. The images are treated, which may include editing unwanted data points or 
distortion, making line fixes, and colorizing, among other processes. The images may also 
be registered to remove lateral motion. Thus, a clean, two-dimensional graphic 

25 representation of the instructor wearing appropriate equipment, such as clothing and gear 
for performing the motion technique, is created to form the animation unit. In alternate 
embodiments, any representation of the animation information, such as an animation data 
file or remotely accessible animation data, may be created for later viewing by a user. 

Fig. 2A is a block diagram of the individual elements that make up the 
30 major element 102 in Fig. 1 . Referring to Fig. 2A, in step 202 the MOCAP cameras are 
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set up. In step 204, the instructor is fitted with a MOCAP suit or other equipment 
facilitating motion capture. Preferably, the instructor is fitted with a body suit for 
capturing motion, such as one including light reflecting, optical balls, approximately 1 inch 
in diameter, placed on the suit at key joint locations such as the elbow, knee, neck, etc. In 

5 step 206, the instructor performs the physical activity in the motion capture environment, 
for example the approved motion techniques outlined in an approved script. If such a 
script is used, it may be written with the cooperation and approval of the governing body 
of representatives or other appropriate organization that have joined together to promote 
and grow the activity or sport. The script comprises the steps that the instructor performs 

10 to teach the particular technique in the said motion capture environment. 

Preferably, the motion capture environment includes light sensitive, optical 
cameras, computer hardware and the software to control the environment, but may include 
other combinations of equipment. One example of such an environment is a system 
provided by Motion Analysis Corporation. Other systems and methods of capturing 
15 motion may be used. In alternate embodiments a sanctioned script need not be provided; 
for example, an actor or instructor may simply execute a physical activity, such as jumping 
rope or performing therapeutic exercises according to a physical therapy regimen. In step 
208, motion data points are generated. In step 210, the system and software generates 
HTR motion data from the data points captured. 

20 Fig. 2B is a block diagram of the individual elements that make up the major 

element 106 in Fig. 1. Referring to Fig. 2B, in step 212, a three-dimensional character is 
created. The character may correspond in physical likeness to the person whose image has 
been captured, or alternately may differ in such physical likeness. In step 214, HTR 
motion data is applied to the three-dimensional character model. In step 216, a 3-D motion 

25 sequence is generated. In step 218, vector graph data is rendered from the 3D motion 
sequence in known ways. 

Fig. 2C is a block diagram of the individual elements that make up the major 
element 1 10 in Fig. 1. Referring to Fig. 2C, in step 220, the two-dimensional vector data 
files are imported into a vector-based animation software program. In step 222, the images 
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are colorized and smoothed. In step 224, the~uhages are also registered to remove lateral 
motion. 

Fig. 2D is a block diagram of the individual elements that make up the 
major element 114 in Fig. 1. Referring to Fig. 2D, in step 226 interactive interface 

5 controls are applied. In step 228 explanatory or other materials may be added. In step 230 
the explanatory materials may be associated with the relevant segments. In step 232 hot 
links to the animation unit may be added to provide highlighted areas of the animation 
sequence display, and, preferably, additional explanatory materials are added. In step 234 
the animation unit may be exported. The animation unit provides instructional information 

10 for a motion sequence to a user. 

The motion technique sequence which is thus created may be divided into 
individual motion segments or subsequences, each which may correspond to portions of an 
approved script or to logical breaks in the motion sequence. For a given motion sequence 
to be taught, several individual motion segments may be performed in sequence. For 
15 example, segments may correspond to a person tossing a ball in the air, drawing a racquet 
back, moving the racquet to strike the ball, and following through with the racquet. 

Explanatory materials, such as text, graphics, audio or other material may 
be associated with each motion segment. In addition, certain areas of the representation 
may be linked to additional content. For example, a typical Internet hyperlink may be used 

20 to associate a portion of the representation with such additional content. A user may 

indicate or request that more information is desired about an object (e.g., a body part or a 
piece of clothing) shown in the animation by, for example, moving an on screen indicator, 
such as a cursor, over the object and possibly clicking on the object using, for example, a 
pointing device such as a mouse or trackball. Additional information, such as the name of 

25 the object or a brief description of the desired position of the object (e.g., the ankle is 
angled inward), may appear if a user drags a pointer over the object. Clicking on the 
object may also display additional information ■- for example, an opportunity to purchase 
the object if the object is an item of equipment. Methods of associating interactive 
information such as labels or hyperlinks with displayed images and objects are known. 
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In the example provided, if a user of the interactive training interface clicks 
on the in-line skates of the digital representation, that user might be brought to a Web site 
provided by a manufacturer of in-line skates who wishes to advertise. Alternatively, 
content explaining the correct fit for in-line skates or what features of performance a user 
5 should look for when purchasing in-line skates may be shown to the user. 

In another example, involving a baseball pitch, a user may click on the 
throwing hand of the digital representation of the trainer. If the user clicks the hand at the 
beginning of the motion sequence the user may be shown a close-up of the grip of that hand 
around a baseball. If the user clicks the hand at the release point in the motion, the user 

10 may be shown a slow motion close-up of the wrist flick being taught at the release point. 
Of course, it will be readily apparent to those of ordinary skill of the art, that any sort of 
additional information content may be associated with portions or objects of the digital 
representation at any time during the motion sequence. Allowing a user to control which 
objects need expanded explanation filters and limits the amount of information provided to 

15 a user, allowing for more effective training. Such methods are not easily effected with 
more traditional physical training, such as live classes, videotapes or books. 

The animation unit also includes control structures which allow the 
animation unit to be affected by the user interface. For example, an index may be 
provided in the animation unit which determines the start frame and stop frame of the 

20 individual motion segments or subsequences within the complete motion sequence and may 
associate those segments or subsequences with particular explanatory material or other 
content. Typically, the animation unit also contains additional control information for 
performing other functions, such as tracking a particular user's viewing of that animation 
unit and/or monitoring a user's progress through a number of individual motion sequences 

25 or subsequences which may be contained in a course of related lessons. 

The interactive control interface template is a software program adapted to 
control the display of the animation unit. Therefore, it includes a portion for displaying the 
motion sequence. Additional video control functions will appear in the control interface 
template, typically arrayed and applied around the motion animation sequence viewing 
30 portion. Other content such as text, Internet hyper-links, additional still images, and audio 
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comprising, for example, explanatory material associated with each motion segment within 
the motion sequence, may be embedded into the interactive control interface, making a 
single, self-contained software program for playing any animation unit on a computer. In a 
preferred embodiment, the interactive interface works in conjunction with an Internet 
5 browser, and utilizes the internal functionality of the Internet browser to perform many of 
the display tasks and file management tasks such as retrieving the animation unit from a 
remote Internet Web Site. Of course, it will be easily appreciated by those of ordinary 
skill of the art, that the interactive interface can be adapted for use on any sort of 
computing system, such as a personal computing device having any sort of configuration, a 
10 television, a hand-held computer device, or any other device capable of running a software 
program and providing a display. 

In one embodiment, the control user-interface contains software elements 
allowing the user to view the instruction techniques from several different views or camera 
angles. This allows the user to have the best vantage point for learning the motion 

15 technique being displayed. One example of benefiting from the ability to view different 
camera angles is the stride technique for in-line skating shown. In Fig. 5A, the stride 
motion technique is shown from the front view, where the user cannot view how far apart 
the instructor's legs are from each other, or how far out from the center of the instructor's 
body the legs are. By viewing the stride technique from a different camera angle, such as 

20 the front view in 7B, the user learns the proper leg positions preformed by the instructor. 

Fig. 6 is the main control user-interface illustrating the last frame from the 
first individual instructional segment and highlighted areas of focus as it can be displayed 
on a monitor from a side camera view according to one embodiment of the present 
invention. Referring to Fig. 6, the control user-interface enhances the learning experience 

25 for the user by displaying visual cues, such as highlighting selected extremities on the 
instructor's body or equipment. For example, highlights may be applied to the animation 
sequence along the instructor's hands, knees, the bend in the elbows, or the position of the 
wrists. These visual cues provide the user with visual assistance in matching 
corresponding extremities or joints of the user's body as the motion dynamically progresses 

30 through the sequence. 
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Fig. 3 is a diagram illustrating the individual instructional segments that 
make up an animation sequence to be displayed on a monitor according to one embodiment 
of the present invention. Referring to Fig. 3, in step 302 the user begins the animated 
sequence. In step 304, the skater is shown skating continuously. In step 306, Step 1 of the 

5 animation sequence (corresponding to segment or subsequence 1) begins. In Step 308, the 
animation of the skater moves to dynamic ready position. In step 310, Step 2 of the 
animation sequence (corresponding to subsequence 2) begins. In step 312, the skater starts 
a right A-frame turn. In step 314, Step 3 of the animation sequence (corresponding to 
subsequence 3) begins. In step 316, the animated skater rotates its head, upper body, and 

10 right knee. In step 318, the animated skater applies pressure to the inside edge of the left 
skate. In step 322, Step 4 of the animation sequence (corresponding to subsequence 4) 
begins. In step 322 the animated skater pulls its heels together and stands to end the stop. 
In step 326 the animation sequence ends. Any individual subsequence may be animated, 
and the animation of such subsequences may be controlled by being played forwards, 

15 backwards, at various speeds, in stop frame form, or by using other methods. The user 
may play each subsequence without playing any other part of the overall animation, starting 
at the beginning of the subsequence and ending at the end of the subsequence. 

For purposes of illustration, a user's interaction with the animation unit will 
now be described. The animation unit will be accessed by a software player which may 
20 show the animation sequence, individual segments, explanatory text and certain control and 
other interface elements on a computer monitor, for example, or as part of another 
computer program, such as a browser. It will be readily understood that other means for 
displaying and controlling the animation unit may be used. 

With reference to Fig. 4A, a computer monitor shows an introductory page 
25 showing a topic 402, in this case, In-Line Skating Techniques, and subtopic 404, in this 
case, braking lessons. An active display panel 406 may show various animation sequences 
related to subtopic 404, for example, standard heel stop, cuff-activated heel stop and grass 
stop. The user may select a particular animation sequence by clicking on the appropriate 
portion of active display panel 406 to form a highlight portion 408. A static display panel 
30 410 will typically show text associated with highlight portion 408 or other explanatory 
materials as described herein. 
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Typical control and navigation elements will also be provided, such as a 
home button 412, a back button 414 and a next button 416. Upon hitting next button 416, 
for example, the user may be shown additional introductory or explanatory materials 
related to highlight portion 408. In this case, active display panel 406 may show a live- 
5 action video, as shown in Fig. 4B, and static display panel 410 may provide additional 
information links 418, such as hyperlinks and the like, to additional explanatory materials. 
Selecting any of information links 418 may provide additional information to the user, 
either in static display panel 410, active display panel 406 or by opening an additional 
browser window, and the like. 

10 After reviewing this additional material, the user will typically press next 

button 416 to enter the lesson itself. 

Referring to Fig. 5 A, each motion technique may be divided into a number 
of smaller, logical instructional units, segments or subsequences. This allows the student 
to have the ability to isolate an individual instructional segment and replay it using the slide 

15 control. The user can view an instructors' motion techniques against a moving background 
{e.g. , a wall or a traffic cone) to give the user the sense of the rate of speed of the image of 
the instructor as it performs the techniques. The user can control different aspects of the 
animation by a known software element such as a slide control 800 or another element. 
The user can view the entire motion technique or view the motion techniques one frame at 

20 a time. In one embodiment, each mark on the slide control 800 corresponds to one frame 
of the instructional animation. Preferably, the user can also control the frame rate of speed 
of the individual motion techniques with the slide control 800. As the animation proceeds, 
instructional information corresponding to the portion of the animation which is being 
displayed may be shown on a portion of the screen. Such instructional information may be 

25 included in the animation unit or animation file containing animation information. 

Referring to Fig. 5 A, active display panel 406 will show a looped animation 
segment associated with the start phase of the lesson. Static display panel 410 will show 
explanatory materials associated with the start phase of the lesson. The control interface 
will provide for separate camera angles to view the animation segment in active display 
30 panel 406, such as side view 420 and front view 422. Again, additional control buttons 
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linking to additional information may be provided, such as a see more button 424 and a 
hear more button 426. Also, typical motion control buttons may be provided, such as a 
rewind button 428, a pause button 430, a stop button 432 and a fast forward button 434. 
Fig. 5B and Fig. 5C show additional frames in the looped animation segment. 

5 Static display panel 410 may include tabbed portions 436 providing direct 

access to individual segments of the sequence, for example, step one, shown in Fig. 6. In 
this case, static display panel 410 will show explanatory materials associated with the 
segment identified as step one and active display panel 406 will show only the frames of 
the animation sequence associated with that segment. The animated character shown in 
10 active display panel 406 may also show highlighted regions 438 which are links to 

additional information. Upon selecting a highlighted region 438, the user may be shown 
such additional information. 

Fig. 7A shows the segment identified as step two from a side camera 
perspective, and Fig. 7B shows the same frame from a front camera perspective. 

15 Fig. 8A shows the segment or subsequence identified as step 3, and in 

particular, the initial frame of that segment or subsequence. Slider 800 shows a segment 
frame region 802 associated with the frames of the segment or subsequence, and a frame 
marker 804 associated with the particular frame being shown. Fig. 8B and Fig. 8C show 
later frames of the segment, as shown by frame marker 804. Finally, Fig. 8D shows the 

20 final frame of the segment from a side camera perspective, and Fig. 8E shows the same 
frame from a front camera perspective. By interacting with slider 800, the user can select 
any specific frame from the segment for closer study, and by choosing different camera 
views, can see that frame from different perspectives. 

Fig. 9 A shows the segment identified as step four, with static display panel 
25 410 displaying explanatory materials associated with that segment. Fig. 9B and Fig. 9C 
show additional explanatory material in static display panel 410 associated with the user 
selecting highlight regions 438. 

Fig. 10A shows explanatory material associated with the end or completion 
of the lesson. .In this case, static display panel 410 may also include additional information 
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links 418. Fig. 10B and Fig. IOC show additional explanatory material in active display 
panel 408 associated with additional information links 418. 

The above-mentioned introductory page, shown in Fig. 4A, may also 
include links to information other than lessons, such as a link to equipment useful for that 
5 topic. Fig. 11A shows the explanatory material in static panel 410 for such information, 
and active display panel 408 will typically show the animation character, including 
highlight regions 438 which link to information, such as the information shown in Fig. 1 IB 
and in Fig. 11C. 

In one embodiment of the system and method of the present invention, an 
10 animation software unit is made up of one or more instructional motion sequences or 

subsequences embedded into a software unit. A software element known as a drop-down 
or pick list allows the user to have control over which instructional lesson he or she wants 
to view and practice. Most students learn the basic fundamentals and then go on to learn 
the intermediate and advanced motion techniques at their own pace and time schedule. 
15 Fig. 12 is a block diagram illustrating the hierarchy and navigational flow for the 

instructional content according to one embodiment of the present invention. Referring to 
Fig. 12, in step 702, the user selects the "Lessons" option. From this step, the user may 
select a number of sports, for example in-line skating (704), golf (706), physical therapy 
(708) or other activities (710). If the user selects in-line skating (704), the user may be 
20 presented with a set of skills comprising in-line skating; for example braking (712), 

aggressive skating (714), or other options (716). If the user selects braking (712), the user 
may be presented with various types of braking lessons; for example the standard heel stop 
(718), the grass stop (720), the spin stop (722) or other methods of stopping (724). 

Fig. 13 is a block diagram illustrating the individual components of the 
25 animation unit according to one embodiment of the present invention. Referring to Fig. 
13, an animation unit (802) according to one embodiment of the present invention includes 
various files (804), including files containing general information (806) and motion 
sequence information (808). The motion sequence information (808) includes motion 
segments (810), which in turn comprises explanatory material (812) and animation frames 
30 (814). The explanatory material (812) and animation frames (814) are related and may be 
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operated to interact with each other. In other embodiments, an animation unit may be of 
different structures, and may include different data. 

In an exemplary embodiment a software display program using the 
animation unit as an input provides a user interface allowing the user to view and control 

5 the animation contained within the animation unit. The software display program may 
interface with an Internet browser, television, or any hand-held computer device to display 
and control the animation. The software display program accepts and processes user input 
to control the digitized animation motion sequence, individual motion segments, the speed 
of animation playback, and one or more different camera views of the same motion 

10 sequence. The software display program may also allow a user to access the additional 
information provided with the animation by, for example, using a pointing device to 
manipulate a cursor to indicate an object for which additional information is desired. 

Accordingly, the present invention provides a process for capturing three- 
dimensional motions of an instructor and converting the three-dimensional motion 

15 information into a three- or two-dimensional digital display format; developing a digital 
character including equipment appropriate for the motion being taught and having the same 
physical characteristics, height, weight, and shape of the instructor; dividing the full 
training motion sequence into individual motion segments; compressing the motion 
segments into a single, digitized software animation unit; and adding textual, audio or 

20 visual explanatory material associated with a particular motion segment to the animation 
unit. The present invention also relates generally to a computer software program that 
retrieves the animation unit from a remote file server or a local database or other location; 
and controls the animation unit to display the training motion, individual motion segments 
and/or the explanatory material on a display, or, for example, an Internet browser, 

25 television, or any hand-held or other computer device capable of running the software 
program and providing a display. 

It will thus be seen that the objects set forth above, among those made 
apparent from the preceding description, are efficiently attained and, since certain changes 
may be made in carrying out the above method and in the system set forth without 
30 departing from the spirit and scope of the invention, that it is intended that all matter ; 
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contained in the above description and shown in the accompanying drawings shall be 
interpreted as illustrative and not in the limiting sense. The present invention may be 
implemented in different manners and used with different applications. 

It is also to be understood that the following claims are intended to cover all 
of the generic and specific features of the invention herein described and all statements of 
this scope of the invention which, as a matter of language, might be said to fall there 
between. 
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CLAIMS 

What is claimed is: 

1. A method of displaying to a user instructional information for a physical 
activity on a computing system, the method comprising: 

5 displaying an animated sequence of a physical activity and associated 

explanatory material, the animated sequence being divided into a set of subsequences which 
may be displayed continuously as the animated sequence or individually as individual 
animated subsequences, the explanatory material being divided into a set of portions, each 
portion associated with one of the subsequences, the animated sequence including images 
10 of objects which may be selected by a user; and 

while an object is selected by a user, displaying to that user additional 
information for that object. 

2. The method of claim 1 further comprising allowing the user to select a 
subsequence and to display the selected subsequence and the associated portion of 

15 explanatory material such that the resulting animation display does not begin before the 
start of the selected subsequence and does not end after the end of the selected subsequence 
and unassociated portions of explanatory material are not shown. 

3. The method of claim 1 wherein the selection of an object is effected by a 
user operating a pointing device to move a cursor over an object. 

20 4. The method of claim 1 wherein the object is one of a representation of a 

human body part and a representation of an item of equipment. 

5. The method of claim 1, comprising allowing a user to alter the camera angle 
at which the animated sequence is displayed. 

6. The method of claim 1, comprising allowing a user to alter the speed of the 
25 display of the animated sequence. 

7. The method of claim 1, comprising allowing a user to alter the time 
direction of the display of the animated sequence. 
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8. The method of claim 1 , comprising providing information links in the 
explanatory material and displaying items of instructional information in response to the 
user's selection of the links. 

9. The method of claim 1, wherein the physical activity to be performed is 

5 performed in accordance with a script and each subsequence is associated with a step in the 
script. 

10. A system for displaying to a user instructional information for a physical 
activity, the system comprising: 

a computer module for displaying an animated sequence of a physical 
10 activity and associated explanatory material, the animated sequence being divided into a set 
of subsequences which may be displayed continuously as the animated sequence or 
individually as individual animated subsequences, the explanatory material being divided 
into a set of portions, each portion associated with one of the subsequences, the animated 
sequence including images of objects which may be selected by a user; and 

15 a computer module for receiving user input allowing a user to select an 

object and, while the object is selected, displaying additional information for that object. 

1 1 . The system of claim 10 wherein the computer module for receiving user 
input allows the user to select a subsequence and causes the computer module for 
displaying the animated sequence to display the selected subsequence and the associated 

20 portion of explanatory material such that the resulting animation display does not begin 
before the start of the selected subsequence and does not end after the end of the selected 
subsequence and unassociated portions of explanatory material are not shown. 

12. The system of claim 10 wherein the selection of an object is effected by a 
user operating a pointing device to move a cursor over an object. 

25 13. The system of claim 10 wherein the object is one of a representation of a 

human body part and a representation of an item of equipment. 

.14. The system of claim 10, wherein the user may alter the camera angle at 
which the animated sequence is displayed. 
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15. The system of claim 10, wherein the user may alter the speed of the display 
of the animated sequence. 

16. The system of claim 10, wherein the user may alter the time direction of the 
display of animated sequence. 

5 17. The system of claim 10, comprising an animation unit providing animation 

information to the computer module for displaying the animated sequence. 

18. The system of claim 10, wherein the computer module for displaying an 
animated sequence may display information links in the explanatory material and may 
further display items of instructional information in response to the user's selection of the 

10 links and the computer module for receiving user input allows the user to select a link in 
the explanatory material. 

19. A method of creating a training animation unit comprising: 

recording a moving image sequence of a live person demonstrating a 
physical activity; 

15 converting the moving image sequence into a series of datapoints 

representing the moving image sequence; 

forming, from the firstseries of datapoints, an animation sequence 
corresponding to the moving image sequence; 

forming, from the animation sequence, at least one vector data file; 

20 registering the at least one vector data file to remove lateral motion; and 

forming an animation data file including a training animation. 

20. The method of claim 19, wherein the step of forming the animation sequence 
includes coloration and adding images of equipment. 

21. The method of claim 19, wherein the step of forming the animation sequence 
25 includes the step of forming a three dimensional digital character. 
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22. The method of claim 19, comprising adding additional explanatory 
information to the animation data file, wherein the additional explanatory information may 
be displayed at the option of a user viewing the training animation. 

23. The method of claim 22, wherein the additional information includes links 
5 which allow a user viewing the training animation to view further information. 

24. The method of claim 19, wherein the animation data file includes selectable 
object representations and where a user viewing the training animation may view further 
information regarding a selected object by selecting such object while viewing the training 
animation. 

10 25. The method of claim 22, wherein the additional information includes items 

of instructional information, each item of instructional information corresponding to a 
portion of the training animation. 

26. The method of claim 19, wherein the training animation includes moving 
representations of an animated person. 

15 27. The method of claim 26, wherein the animated person has the same 

characteristics as the live person demonstrating the physical activity. 

28. The method of claim 19, wherein the training animation includes a plurality 
of views, each from a different viewpoint. 

29. The method of claim 19, wherein the training animation data file is stored in 
20 a database. 

30. A method of creating a training animation sequence comprising: 

recording a live person demonstrating a physical activity and forming a 
series of datapoints corresponding to the demonstration; 

forming, from the series of datapoints, an animation data set corresponding 
25 to the demonstration; 

. including in the animation data set additional information corresponding to 
certain objects animated in the animation data set; and 

forming a training animation sequence from the animation data set. 
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31. The method of claim 30, wherein the step of forming the animation data set 
includes coloration and adding equipment. 

32. The method of claim 30, wherein the step of forming the animation data set 
includes the step of forming a three dimensional digital character. 

5 33. The method of claim 30, comprising adding additional information to the 

training animation sequence, wherein the additional information may be displayed at the 
option of a user viewing the training animation sequence. 

34. The method of claim 33, wherein the additional information includes 
hyperlinks. 

10 35. The method of claim 33, wherein the additional information includes 

information regarding selectable objects which are animated in the training animation 
sequence and the information is displayed when the object is selected by a user viewing the 
training animation sequence. 

36. The method of claim 33, wherein the additional information includes items 
15 of instructional information, each item of instructional information corresponding to a 

portion of the training animation sequence. 

37. The method of claim 30, . wherein the training animation sequence includes 
moving representations of an animated person. 

38. The method of claim 37, wherein the animated person has the same 
20 characteristics as the live person. 

39. The method of claim 30, wherein the training animation sequence includes a 
plurality of views, each from a different viewpoint. 

40. The method of claim 30, wherein the training animation sequence is stored 
in a database. 

25 41. The method of claim 30, wherein the training animation sequence may be 

distributed to and accessed by a user. 
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Figure 2C 
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Figure 3 
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Figure 5A 
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Figure 5C 
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your toes point outward and spin 
counterclockwise to a stop. 
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Figure 6 
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From a stride, move into the 
ready position with your weight 
evenly distributed. Your shins 
should be pressed into your 
skate tongues in a forward lean. 
This gives you better stability 
and control. 
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Figure 7 A 
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Start an A-frame turn to the right, 
lifting your right heel. 
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Figure 7B 
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Start an A-frame turn to the right, 
lifting your right heel. 
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Figure 8C 
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Rotate your head, upper body 
and right knee out, shifting 
pressure to the inside edge of 
your left skate. As soon as your 
right heel is under your body so 
that your toes point outward, 
quickly return your whole skate 
to the pavement and balance 
your weight evenly between both 
skates. Spin counter-clockwise. 



406v 

PR A * 



802^804^ 



800 



420 



CAMERA 

o 



<lllllli:6S:!!!i:i!!l!!]llllll! 



TOP 



SIDE 



422- 



FRONT 



424 



426 



See More 



412 
414 



' ^HOME^ ^BACK^ ^NEXT^ ^ 



^410 
416 





CTO 428-* 



Hear More 



*o m □ c> 

430^_^ 4 32^ 434^ 



Figure 8D 
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Rotate your head, upper body 
and right knee out, shifting 
pressure to the inside edge of 
your left skate. As soon as your 
right heel is under your body so 
that your toes point outward, 
quickly return your whole skate 
to the pavement and balance 
your weight evenly between both 
skates. Spin counter-clockwise. 
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Figure 8E 
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Rotate your head, upper body 
and right knee out, shifting 
pressure to the inside edge of 
your left skate. As soon as your 
right heel is under your body so 
that your toes point outward, 
quickly return your whole skate 
to the pavement and balance 
your weight evenly between both 
skates. Spin counter-clockwise. 
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Figure 9 A 
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Pull your heels together and 
stand up to stop fully. 

You stop within roughly half a 
rotation. 
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Figure 9B 
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On center edge of skates. 
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Figure 9C 
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Heels together. 
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Figure 10A 



402-~> 404—^ 
BRAKING LESSON : Spin Stop 



406v 




CONGRATULATIONS! 

You've finished this lesson. 

Stuff you should do next: 

A18 /f9MIIS Prills 
* Troubleshooting 
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Congratulations! 

You're through the lesson. Here are 
some drills you can practice and some 
troubleshooting tips to help you get 
completely comfortable with your new 
skill. 

If you still have questions we haven't 
answered, check out the in-line bulletin 
board. Our instructors, each one 
certified from the International In-line 
Skating Association, will be happy to 
help you out. 



Figure 10B 
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Skills Drills 



s While standing in one place, lift 
the heel of your right skate. 



Practice the A-frame turn. 



Figure 10C 
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Troubleshooting 



s Avoid over-rotating by slowing 
down the speed at which you enter the 
A-frame turn. 

s If you are off balance, check 
your hand position. They should be in 
front of you at waist level throughout 
the turn. 

S To get the tight turning arc you 
need for best maneuverability and 
control, lean forward with bent knees as 
you enter your turn and apply pressure 
to the inside edges of your skate. 
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Figure 11 A 



IN-LINE SKATING : Equipment 



Skating Equipment 

In-line skating starts with 
protective gear. The basics are 
wrist guards, kneepads, elbow 



pads and a helmet Certainly you 
will need skates and there is a 
huge variety from which to 
choose. 

For more information, click on 
the figure to the right. 
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Figure 11 B 



Helmet 

Your helmet must be durable, fit 
securely with a buckle and allow 
you to slide if you fall. Look for 
an ASTM, SNELL or ANSI rating. 



Figure 11C 



In-line Skates 

There are in-line skates designed 
specifically for hockey, freestyle 
skating, racing, fitness training, 
recreation, aggressive skating 
and cross training for sports 
such as skiing. 



WO 01/09861 



15/16 



PCT/USOO/20752 



IN-LINE 
SKATING 704 



BRAKING I 
712 J 



STANDARD 
HEEL STOP 
718 



Figure 12 



LESSONS 702 



3 



GOLF 706 



f\ PHYSICAL 



THERAPY 
708 



AGGRESIVE 
714 



I OTHER 71 6 I 



GRASS 
STOP 720 



SPIN STOP 
722 



OTHER 710 



3 



OTHER 724 



WO 01/09861 



16/16 



PCT/US00/20752 



Figure 13 
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Start the spin stop by striding 
into the ready position and 
gliding at a moderate speed. 
Move into the beginning of an A- 
frame turn to your right Raise 
the heel of your right skate, 
rotate your right knee out so that 
your toes point outward and spin 
counterclockwise to a stop. 
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different views (420, 422) of the same animation motion sequences. An interactive computer-based user-interface allows a student 
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